INSTITUTIONAL EVALUATION: 
CAN IT CONTRIBUTE TO 
IMPROVING UNIVERSITY 
PERFORMANCE? 


Australian universities, like their counterparts in 
other western countries, are increasingly operating 
under financial constraints which have accentuated 
the need for improved planning and administration. 
These financial pressures have stimulated interest 
in techniques for assessing system, institutional, 
and sub-unit performance. Following the report of 
the Williams Committee with its recommendations 
for extending research into institutional and system 
performance', and the establishment of the Evalua¬ 
tive Studies Program by the Tertiary Education 
Commission 2 , most universities are engaging in 
evaluative studies intended to yield information 
which will assist endeavours to improve institutional 
performance. Some of the Australian and overseas 
experience with institutional evaluation and review 
has been reported by Clarke and Birt 3 , Holdaway 4 , 
Calvert 5 , and Harman and Johnston 6 . 

However, there are substantial obstacles confront¬ 
ing these endeavours. First, the basic institutional 
processes are not well understood and so the mean¬ 
ings of the terms “performance" and “management" 
in the university setting are not completely clear. 
Hence, it is not possible to define exactly what is to 
be evaluated. Second, adequate measurement 
techniques and judgement procedures are not 
available and so there is a substantial gap between 
what is desirable in an evaluation and what is 
achievable. Third, the results from evaluative stu¬ 
dies are generally not in the form that can be readily 
utilized by decision-makers in attempts to improve 
performance. Current evaluative studies often pro¬ 
vide results which are fragmented assessments 
relating to a set of arbitrary dimensions of “perfor¬ 
mance”. The dimensions are generally only tenu¬ 
ously related to any model of institutional operation 
or to notions of effectiveness and efficiency. Hence 
the assessments do not provide much guidance in 
choosing among alternative courses of action 7 . This 
paper examines the problems involved in assessing 
the performance of universities and their sub-units 
(faculties, schools or departments), and the 
strengths and weaknesses of the evaluation tech¬ 
niques available. The starting point for this exami¬ 
nation is a consideration of the basis for evaluation 
provided by Chapter 18 of the Williams Report. The 
limited foundation provided by the Williams Report 
is supplemented by drawing upon the relevant 
research literature in an attempt to contribute to the 
provision of a sound basis for evaluation 
endeavours. 
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The purposes and uses of evaluation 
Chapter 18 of the Williams Report deals specifically 
with one of the items in the terms of reference which 
directed the Committee to advise on the “means of 
evaluating the quality and efficiency of the system". 
The Committee associated the increasing concern 
with evaluating quality and efficiency with the 
greatly increased financial burden of supporting 
education, the allegations of mismatches between 
supply and demand for graduates, and the failure of 
education to achieve desired sociat objectives. The 
Committee supported its recommendations for the 
extension of evaluative studies by a conceptual 
analysis and suggestions tor evaluation checklists 
relating to various components of the system. The 
Committee reported that fewer submissions were 
received in relation to this item of the terms of refer¬ 
ence than in relation to any other, and acknowl¬ 
edged that: 

interest in evaluating the quality and effi¬ 
ciency of the education system has grown 
faster than the capacity to do the evaluation. 

The Committee's review of current arrangements 
for evaluation reveals the lack of both evaluative 
focus and rigour in the current scattered and dis¬ 
jointed evaluation practices. Unfortunately, the 
Committee’s own discussion of the topic is superfi¬ 
cial; it draws upon no theoretical or empirical foun¬ 
dation, and there is a lack of rigour in its analysis of 
the concepts and problems. 

One deficiency in the Report is the absence of any 
discussion of what is meant by “evaluation" and the 
ways in which evaluation is distinguished from 
other decision-making activities. Evaluation is a 
complex process as the substantial literature on 
educational evaluation attests. Feaslby 8 has recently 
identified nine different uses of the term “evalua¬ 
tion". The first is to regard evaluation as measure¬ 
ment and to focus on the data and the formalised 
instruments and standardised scales involved in 
producing results that are comparable and replica¬ 
ble. By contrast, evaluation may be seen as profes¬ 
sional judgement in which visitation teams and peer 
review provide the means of judgement. A different 
focus is to regard evaluation as a comparison of 
actual outcomes with the objectives or intended 
outcomes. Alternatively, there is “goal-free" or 
responsive evaluation which attempts to look at all 
outcomes including the unintended effects. Evalua¬ 
tion may also be regarded as a political process 


aimed at conflict resolution through negotiation. A 
related notion is to use evaluation as a means of 
reducing complacency or as a means of achieving 
changes. Following on from this, evaluation may be 
regarded as a process designed to produce useful 
information for decision-making in relation to 
attempts to improve the system or programme. 
Finally, evaluation may be undertaken merely as a 
ritual to provide a picture of rationality and accoun¬ 
tability which promotes a feeling of security. 

Quality and efficiency 

The basis of the Williams Committee's discussion of 
evaluation is an analysis of the terms "quality" and 
“efficiency”. The Committee’s contribution was to 
provide dictionary definitions of these terms. Qual¬ 
ity is defined as a "degree of excellence”. Since the 
complex concept of "excellence” is not analysed, 
the meaning of quality is not greatly clarified or 
specified. There is a considerable literature on edu¬ 
cational quality which the Committee could have 
usefully drawn upon. Some of the most recent work 
includes Lawrence and Green 9 , Lawrence and Sol- 
mon’°, and Kith". Astin 12 reviews the “ five very dif¬ 
ferent concepts of quality that have dominated 
educational research and policy during the past 
several decades". One view is the “mystical” con¬ 
ception which maintains that quality 

simply cannot be defined or measured 
because the activities of institutions are too 
complex and varied, because different institu¬ 
tions have different objectives, because the 
outcomes of higher education are too subtle, 
because methodological problems are insur¬ 
mountable, and so on. 

A second view is the “reputational” conception of 
quality as coinciding with the level of prestige 
accorded by an opinion survey. A view promoted by 
educators is to equate quality with an institution’s 
educational resources of “bright students, highly 
trained and prestigious faculty, and affluence". An 
increasingly popular approach is to relate institu¬ 
tional or system quality to the outcomes produced. 
A major difficulty with this view is that the level of 
outcomes may not necessarily result from institu¬ 
tional or system impact. This leads to the “value 
added” view which defines quality in relation to the 
impact on students; that is, the institution's contri¬ 
bution to their intellectual and personal develop¬ 
ment. The value-added approach directly relates 
quality and excellence to effectiveness. A quality 
institution or system is one that makes an impact; it 
is effective in promoting student learning. Quality or 
excellence in this sense can be applicable to any 
level of education. This contrasts with the usual 
meaning of the “pursuit of excellence” which relates 
excellence to an institution’s involvement in the 
forefront of intellectual endeavour. Under a value- 
added approach a high quality institution is not 


necessarily one where students achieve a high gra¬ 
duation standard but rather where students have 
been lifted substantially from their entry standard. 

The Committee’s discussion of efficiency is even 
less satisfactory than that of quality. Rather than 
draw upon the literature in the fields of economics 
and management in which the usefulness of distin¬ 
guishing effectiveness and efficiency is well estab¬ 
lished 13 , the Committee defined efficiency as a 
“degree of effectiveness with which something is 
done”. In classical management terms, effective¬ 
ness refers to the extent of goal achievement; that is, 
the extent to which the actual outcomes match the 
intended outcomes. Alternatively, in organisations 
such as universities where goals are problematic, 
effectiveness can be related to some other criteria 
such as success in obtaining resources or success 
in satisfying major constituencies. Efficiency refers 
to the extent to which the level of output has been 
achieved at least cost. The distinction between 
effectiveness and efficiency is important for evalua¬ 
tion. The evaluation of institutional or sub-unit "per¬ 
formance” can be regarded as having two compo¬ 
nents; assessing the level of effectiveness and 
assessing the level of efficiency. The notion of effi¬ 
ciency and the related one of “productivity”are often 
unwelcome in educational circles' 4 , but it must be 
recognised that assessments of “efficiency” will 
continue to be made with varying degrees of accu¬ 
racy and reliability, and so the basic question is not 
how to prevent them but how to improve them. 
There are many discussions of the meaning of effi¬ 
ciency and productivity in relation to education but 
the definitive works in interpreting these concepts 
in ways that are appropriate and significant in the 
higher education context are Toombs 15 and Wal- 
Ihaus 16 . The notion of efficiency in the Australian 
university context is discussed by Cowen 17 and Lay- 
ton 18 . A review of research on the efficiency dimen¬ 
sion of institutional performance is provided by 
Lindsay 19 . The basic issue underiying the use of 
these concepts is the extent to which the economic 
“theory of the firm", and classical business and 
bureaucratical models for organizations and man¬ 
agement can be usefully applied to universities. 
Cohen and March® and Weick 21 have described an 
organizational model which in many ways provides 
a better representation of university operation than 
the classical models. The main organizational char¬ 
acteristic is ill-defined, complex, conflicting and 
problematic goals. The goals of universities are not 
clear, specific, and agreed. They change over time; 
are different for different groups; are often in con¬ 
flict or the subject of factional conflicts; and are not 
readily translated into clear-cut programmes. The 
other characteristics include unclear and imper¬ 
fectly understood technology; fluid participation in 
decision-making with members taking part only 
intermittently according to their interest in particu¬ 
lar issues; structural looseness with relatively ill- 
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defined links and relatively high leveis of individual 
and sub-unit autonomy; and widely differing criteria 
of success applying simultaneously as participants 
make judgements in the absence of measurement 
or market prices for outputs. 

Evaluating effectiveness and efficiency 
If the classical management perspective is adopted, 
the evaluation of effectiveness involves comparing 
outcomes with goals with the aim of making correc¬ 
tions where outcomes deviate seriously from those 
intended. From this perspective, the actual out¬ 
comes have maximum value when they are identi¬ 
cal with the intended outcomes, This notion of 
evaluation combines two of the earlier conceptions 
of evaluation: evaluation as a comparison of out¬ 
comes and objectives, and evaluation as a basis for 
decisions. If a non-goal-directed organizational 
model is adopted, evaluations of effectiveness in a 
“goal-free" sense can also be undertaken by assess¬ 
ing the value of the actual outcomes in relation to 
other sets of criteria such as those of social utility. 

Selecting or agreeing upon a notion of effectiveness 
is only the first step, however, as there are substan¬ 
tial methodological problems in actually making the 
evaluation. There are many outcomes from the insti¬ 
tutional processes, and, as a consequence of the 
competition among activities for resources, produc¬ 
ing one outcome may only be possible at the sacri¬ 
fice of another. In the educational setting there is no 
agreed mechanism for deciding the trade-offs 
among multiple outcomes. Hence there is no way of 
deciding on an ideal outcome mix or of weighting 
the various outcomes to produce a single compo¬ 
site measure. What, for example, are the relative 
values of a PhD and a Bachelor's degree, or of a 
specific professional degree or a general degree, in 
deciding the trade-offs for an ideal mix of these 
outputs. Lindsay and Bailey 22 have stated this prob¬ 
lem as follows: 

The outputs of higher education arenotsolely 
goods whose value is determined in the 
market place, so the relative value is only par¬ 
tially determined by a price mechanism. Even 
if a market system existed for all the outputs it 
would only provide one way of measuring 
value. The value of the outputs of higher edu¬ 
cation is a complex and normative issue: the 
existence of varying perspectives on the pur¬ 
poses of higher education will result in differ¬ 
ing views about the value of the outputs and 
so many different assessments of perfor¬ 
mance may be made. 

Secondly, many of the outcomes of education are 
intangible and therefore not easily identified or 
measured. Despite the considerable efforts devoted 
to outcomes research in the United States 23 , there 
are still substantial deficiencies in the techniques for 
measuring outcomes. Thirdly, since education is 
only one of the many influences in the development 


of individuals and the progress of society, it is diffi¬ 
cult to isolate education’s distinctive effects. These 
considerations present a major challenge to those 
engaged in evaluating effectiveness in education. 

The influence of education's complexity and intan¬ 
gibility can also be seen in relation to the application 
of the notion of efficiency to educational institu¬ 
tions. The most appropriate concept of efficiency is 
one which requires the comparison of outcomes 
with the resources employed. In relation to effi¬ 
ciency in higher education, several simplistic 
notions are frequently used. One involves judging 
efficiency only in relation to cost, by assuming that 
a lower cost is necessarily preferable to a higher 
cost. For example, an institution with a cost per 
student of $6,000 may be judged as less efficient 
than one with $4,000, regardless of the quality of the 
service provided to the students. Another notion is 
to judge efficiency only in relation to outcomes; that 
is, to assume that improved outcomes are desirable 
regardless of cost. This notion confuses efficiency 
with the notions of quality and effectiveness, A third 
approach is to regard the crude proxy input-output 
measures which are available, such as costs per 
student or student-staff ratios, as adequate mea¬ 
sures of efficiency or productivity and to use them in 
decision-making without regard to their considera¬ 
ble imperfections. A fourth approach is to demand a 
strict cost-benefit accounting of expenditures in 
relation to pecuniary outcomes. While the first two 
notions fail to recognise that efficiency involves a 
relationship between two variables — cost and out¬ 
comes; the third fails to recognise the gap between 
desirable and actual measurement capabilities; and 
the fourth that the non-monetary benefits may well 
outweigh the monetary benefits. 

Thus, the substantial methodological problems 
involved in measuring outcomes hinder the evalua¬ 
tion of efficiency just as they do in relation to effec¬ 
tiveness. There is some danger, however that these 
problems will be ignored or discounted simply to 
suit educational fashion or political expediency. 
The dangers of simplistic approaches to efficiency 
in education have been amply documented 2 ". Until 
the methodology for measuring outcomes is sub¬ 
stantially improved, the assessment of educational 
effectiveness and efficiency will remain only partial 
and the results from any evaluation enterprise 
should be used with due regard to their limitations. 

The unit of analysis 

Different evaluation techniques are required for dif¬ 
ferent units of analysis. Evaluation can take place 
using any unit of analysis from the individual stu¬ 
dent, through the programme departmental and 
institutional levels, to the sector and system levels. 
As the level becomes more aggregated the availabil¬ 
ity of techniques and the amount of previous 
research becomes progressively less. One outcome 
of this has been the rather dubious practice of using 


the techniques designed for one level of evaluation 
at a more aggregated level 25 . Each level is not 
merely an aggregated version of the level below. For 
example, an education system is not simply a 

large accumulation of individuals, curricula 
and projects... Hence an education system 
evaluation must be viewed as requiring a 
number of considerations which are unique 
to and different from those appropriate for the 
evaluation of individuals or curricula ai . 

Also, the notion of "performance” is more complex 
and intangible for the more aggregated levels, and 
universities do not have a developed capability of 
undertaking such evaluations. Despite the current 
pressures, institutions must avoid the temptation to 
transfer the techniques for evaluating individual 
staff members to the departmental and institutional 
levels, or to borrow inapplicable personnel and 
organizational evaluation techniques from the busi¬ 
ness world. 

The Institutional Self-Study Approach 

The specific proposals for evaluation made by the 
Williams Committee are in the form of a number of 
checklists for use in assessing the role of govern¬ 
ments and coordinating bodies, and for assessing 
the performance of institutions. 

The Committee did not make suggestions for eva¬ 
luation at the system or sector levels. Johnstone 27 
provides the most useful introduction to system- 
level evaluation. He suggests the use of a series of 
input, process, and output indicators which are gen¬ 
erally defined as ratio rather than gross variables. 
Useful analyses can then be undertaken by study¬ 
ing trends in the indicators over time within one 
education system or by comparing the levels of 
indicators across a number of systems. Other but 
less comprehensive discussions of educational sys¬ 
tem indicators are provided by the OECD 28 , and 
Rossi and Gilmartin 22 . The Williams Committee did 
make the claim that "the Australian community has, 
however, been well served by its system of educa¬ 
tion'"-' 0 , but did not directly relate this claim to any of 
the data or evidence appearing elsewhere in the 
Report. Even though many existing measures such 
as participation rates by age, sex, level and type of 
study; graduation and attrition rates; real resources 
per student; student-academic staff ratios; and 
student-total staff ratios are examined in the Report 
there is no attempt to discuss the systematic use 
and interpretation of these measures as indicators 
of the system’s performance. 

Useful studies of such indicators over time within a 
system have been previously carried out by Wood- 
hall and Blaug 3 ' in their study of productivity trends 
in the British university system from 1938 to 1962; 
by O’Neill 32 who studied trends in the ratio of credit 
hours to costs in American higher education; by 
Hettich 33 who undertook a study of Canadian uni¬ 


versities along similar lines to that of Woodhal! and 
Blaug; by Harris 3 " in his extensive “statistical por¬ 
trait” of American higher education; and by Willi¬ 
ams 35 in his study of real resources per student in 
Australian universities. 

It must also be recognised that the Williams Com¬ 
mittee’s checklists, which were not intended to be 
comprehensive, are also largely non-operational; 
that is, they cannot be applied and answered with¬ 
out a specification of the meaning of some terms 
used in the questions. For example, the terms “con¬ 
sistent”, “effective", and “reasonable” are used with¬ 
out any specification as to how to judge whether or 
not the particular attribute meets the appropriate 
level. In supporting a checklist approach the Willi¬ 
ams Committee is recommending that institutions 
engage in a particular type of self-study. This type 
of approach, which has a long history in the United 
States in the form of institutional research and 
accreditation assessments, has recently been given 
a great deal of attention culminating in the compre¬ 
hensive evaluation techniques for studying individ¬ 
ual institutions and their sub-units which have been 
proposed by Dressel 36 and Miller 37 . Miller, for exam¬ 
ple, suggests an examination of ten dimensions of 
the academic enterprise: 

® Goals and objectives 
® Students’ learning 
® Faculty performance 
® Academic programmes 

• Institutional support services 
» Administrative leadership 

• Financial management 

• Governing board 
® External relations 

• Institutional self improvement 

For each area a number of criteria in the form of 
questions are suggested; for example, in relation to 
Students' Learning, Miller suggests five criteria: 

® How do students rate their advising and counsel¬ 
ling system? 

® Are retention rates reasonable? 

• Is an array of individualised and compensatory 
learning resources available? 

» How effective is the student affairs administration? 

• Is satisfactory progress evident towards learning 
goals? 

While some attempts have been made to develop 
standardised and reliable instruments for some of 
these dimensions, much of the assessment must be 
based on judgements by the evaluators, who may 
be the institution’s staff or visitation teams involved 
in peer review. As this type of evaluation technique 
does not include any means of specifying the stand¬ 
ards for use In making comparisons or judgements, 
the results are likely to be highly subjective and 
unreliable. Although the use of peer review is an 
attempt to introduce some comparability or stan- 
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dardisation into this process, the American litera¬ 
ture on accreditation and self-study attests to the 
very limited gains in reliability from this strategy 33 . 

A more sophisticated approach has been the devel¬ 
opment of standardised instruments for making 
measurements on some institutional dimensions. 
These instruments include the measures of student 
learning and outcomes developed by the Educa¬ 
tional Testing Service (ETS), the American College 
Testing Program (ACT), and the National Center 
for Higher Education Management Systems 
(NCHEMS); the College and University Environ¬ 
ment Scales (CUES); the Institutional Functioning 
Inventory (IFI); the Institutional Goals Inventory 
(IGI); and the many techniques for analysing costs, 
staff activities, and space utilisation developed by 
NCHEMS and other agencies. A number of useful 
reviews of these instruments are available 39 . For the 
dimensions concerned, these instruments provide 
the most suitable means of assessment, although 
they also are by no means perfect. 

There are a number of difficulties with the self-study 
approach. Questions may be raised in relation to 
how well the dimensions used in the checklists 
represent the performance of an institution or 
department. The inclusion of most dimensions 
appears to be based on commonsense or adminis¬ 
trative folk-lore rather than on the basis of a theoreti¬ 
cal or empirical foundation. Even though institu¬ 
tional processes are not well understood, it is still 
desirable that each dimension be justified in relation 
to some overall concept or model of “institutional 
performance”. One consequence of the absence of 
a theoretical framework underpinning the institu¬ 
tional self-study approach is that the approach does 
not provide any mechanism for weighting the 
assessments for the disparate dimensions and 
aggregating them into an overall assessment of per¬ 
formance. Thus a good deal of responsibility and 
power is given to those who try to synthesize the 
results and judge the overall performance. 

Making separate assessments in relation to each 
dimension means that the institutional self-study 
approach does not necessarily provide information 
about effectiveness and efficiency. To do so, the 
criteria for assessing each of the dimensions which 
concern outcomes must involve, on the one hand, 
comparisons with intended outcomes, and on the 
other, with costs or resources used. Assessment of 
the former type is much more commonly implied in 
the dimensions and criteria proposed; the institu¬ 
tional self-study approach is thus largely concerned 
with assessments of effectiveness rather than 
efficiency. 

Some of the problems of the institutional self-study 
approach may be alleviated if the evaluation of the 
institution is undertaken on a regular basis. 


Because a comprehensive self-study tends to be 
time consuming and costly, it is often undertaken as 
a “one shot" evaluation. However, if the institution is 
studied regularly over a period of time, it may be 
possible to develop a basis for making comparisons 
and judgements in relation to the trends in the insti¬ 
tution’s performance. Trends in such quantitative 
indicators as costs per student and student-staff 
ratios could be usefully studied in this way, as could 
the results from the application of survey and other 
measuring instruments. In addition, if carried out 
frequently, non-quantitative assessments might 
also be useful in identifying trends in performance 
improvement or decline. 

Inter-institutional comparisons 

An alternative approach to institutional evaluation, 
which is most profitable where quantitative mea¬ 
sures or indicators are available, is to make inter- 
institutional comparisons. To the extent that a set of 
institutions is homogeneous, comparisons can 
yield useful assessments of both relative effective¬ 
ness and relative efficiency. Peterson* sees the 
development of the inter-institutional research field 
as being dependent on progress in three areas; 

major standardised instruments that have 
institutional norms and available distribution 
channels, the growth of large-scale institu¬ 
tional data bases and related exchange proce¬ 
dures that make institution-based data more 
available, and the emergence of the computer 
networks with more sophisticated software 
for improved access and ease of data sharing. 

Sizer 4 ’ sees progress as being dependent on the 
establishment of a central organisation to under¬ 
take a series of tasks: 

® establish and agree with participating institutions 
a detailed data element dictionary; 

• design and agree with participating institutions a 
methodology for collecting the data; 

• collect and check the actual data; 

® calculate, tabulate and circulate input and output 
measures and performance indicators to participat¬ 
ing institutions; 

® assist the management of the participating institu¬ 
tions to interpret their data; 

» undertake a continuous programme of education 
in the use of inter-institutional comparisons. 

Provided the relationships between institutions and 
the Commonwealth Tertiary Education Commis¬ 
sion (CTEC) do not deteriorate further, the CTEC 
could continue to expand its involvement in these 
tasks and to involve institutions in the comparative 
evaluation process. Institutions however, may well 
feel threatened by the sharing and publication of 
data, particularly cost information. The central 
organisation needs the confidence of the institu¬ 
tions in order to promote inter-institutional perfor¬ 


mance assessments, but the pervasive lack of trust 
which is in part an outcome of simplistic attacks on 
educational efficiency and the biased use of educa¬ 
tional data for political purposes, will hinder endea¬ 
vours to promote comparative research. 

The way forward 

A number of problems must be solved before insti¬ 
tutional and departmental evaluations can make a 
major contribution to efforts to improve perfor¬ 
mance. A better conceptualisation is required of the 
dimensions of institutional performance and of the 
inputs and outputs to the institutional processes. 
Better measurement techniques and judgement 
procedures are required in order to produce more 
valid and reliable data. To be useful in management 
endeavours, ways must be found to relate the infor¬ 
mation from evaluations directly to effectiveness 
and efficiency which are two factors of major con¬ 
cern to management, To achieve this, ways must be 
found to reconcile and consolidate the often con¬ 
flicting and disparate information into a weighted 
overall assessment. Finally, ways must be found to 
provide information economicaily, so that it is feasi¬ 
ble to undertake continual evaluation as means of 
providing recent and timely information to the deci¬ 
sion makers, 

While all of the available approaches to institutional 
performance assessment have quite severe limita¬ 
tions, the one-shot attempt at comprehensive self- 
study based on conceptually unsound checklists 
appears to be the least satisfactory type of 
approach. It is to be hoped that future evaluation 
endeavours in Australian institutions of higher edu¬ 
cation are directed towards more fruitful activities. 
The study of institutional trends over time, and 
inter-institutiona! comparisons particularly those 
involving quantitative measures seem to have the 
most potential for providing information which is 
useful to management. Since Australian universi¬ 
ties form a relatively homogeneous group, ensuring 
comparability poses less difficulties than for a more 
diverse group. 

One tempting response to the quite severe limita¬ 
tions in current evaluative capability is to use them 
as justification for refraining from any attempts at 
institutional evaluation. The opposite response, 
which is now replacing the first in popularity, is to 
minimize or ignore the limitations in order to suit the 
current educational fashion for evaluation and the 
underlying political purposes. Neither of these 
responses is realistic. The lasting austerity to which 
Australian universities are now adjusting means 
that improving institutional performance and 
managerial capability are pressing needs. Clearly, 
institutional evaluation has a part to play in meeting 
these needs, and in any case, current pressures for 
evaluation, both from within and without the univer¬ 
sities, are now irresistible. On the other hand, ignor¬ 
ing the absence of a sound theoretical and 


empirical basis for the concepts of institutional “per¬ 
formance" and university “management", and for 
the available measurement and judgement tech¬ 
niques, will only result in unproductive activity 
which has a marginal impact on decision-making, 
The appropriate response is a cautious and scepti¬ 
cal application of the best available evaluative tech¬ 
niques giving due consideration to both their 
strengths and limitations. Institutional evaluation is 
still in an embryonic stage; it is under conceptual¬ 
ized and lacks rigorous operational models and 
techniques. While current evaluative activities must 
continue, their implicit assumptions and limitations 
must be recognised and efforts devoted to develop¬ 
ing a more scholarly basis for evaluative endea¬ 
vours. 
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ONE m ALL IN: 
FAUSA AND THE ORIGINS 
OF THE ACADEMIC 
SALARIES TRIBUNAL 

Origins 

In 1974 the Academic Salaries Tribunal was estab¬ 
lished under the Commonwealth Government's 
Remuneration Tribunals Act. It was founded as a 
Commonwealth agency with no state representa¬ 
tion to attempt to resolve two issues: first, whether 
differences in the character and function of CAEs 
and universities implied a difference in the level of 
salaries to be paid to academic staff working at 
institutions in either sector: and second, the need to 
institute a mechanism to provide national guide¬ 
lines for regular and equitable adjustments to aca¬ 
demic salaries. At its inception the Tribunal was 
empowered to determine the salary rates for aca¬ 
demic staff at the Australian National University and 
Canberra College of Advanced Education and to 
report to the Commonwealth Minister for Adminis¬ 
trative Services in relation to academic salary rates 
for State universities and advanced education insti¬ 
tutions. Since that time, while State Governments 
and State-based tertiary institutions have not been 
legally bound by the Tribunal's recommendations, 
they have generally adhered to the academic salary 
rates suggested. As a result, the salaries of various 
categories of academicstaff have remained uniform 
across Australia. 

This was not always the case. It was a series of 
discordant and ad hoc arrangements in the 1960s 
which led to the establishment of the Academic 
Salaries Tribunal as a Commonwealth agency. Dur¬ 
ing that period public policy-making on academic 
salaries did not have the coherence and definition of 
a programme, particularly as the Commonwealth 
Government was sensitive to a possible accusation 
from the States that it was becoming involved in 
’wage-fixing' in universities. 

Throughout the 1960s, with considerable reluc¬ 
tance, the predecessor to the Universities Council 
of the Commonwealth Tertiary Education Commis¬ 
sion the Australian Universities Commission (AUC), 
found itself inextricably involved in attempts to 
resolve the academic salaries issue. Despite its 
moves to adopt a disinterested stance, the AUC had 
to take part, particularly as the component of the 
recurrent grants devoted to academic and non- 
academic salaries by each university was so large. 

In some ways, similar paths were followed in Britain 
and Australia, although parallel developments 
occurred a decade earlier in Britain, and the British 
University Grants Committee (UGC) played a more 
active and sustained rolethan its Australian counter¬ 
part, the AUC. Early in the post-war years Britain 
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established a rough uniformity of academic salar- 
ies. In the 1950s the British Association of University 
Teachers (AUT) sought to negotiate academic 
salaries with the UGC and the Treasury by direct 
bargaining on a trade union model. During thattime 
the UGC maintained a unique and important role in 
the process of reviewing academic salaries. There 
was no workable pattern of negotiation between 
employers and employees and the UGC did not act 
on behalf of universities but in its own right. 

The AUT put up to the government successive 
'claims’ based on the comparability principle used 
by the Civil Service Commission and specifically on 
an attempted direct comparison with civil service 
salaries, arguing that the chief alternative employ¬ 
ment for academics was the administrative or scien¬ 
tific class of the civil service. But this comparison 
was never specifically accepted by the UGC or 
Treasury. A 'stately minuet’ developed whereby the 
UGC listened to the AUT, listened tothe Committee 
of Vice-Chancellors and Principals and then made 
its own confidential recommendations to the 
government'. 

In 1963 the UGC's role was significantly diminished 
when academic salaries were made the subject of 
an independent review by the National Incomes 
Commission. There was then a brief reversion in 
1966 to the arrangement whereby the government 
made a decision after consultations with the UGC, 
which in turn consulted the AUT and the Commit¬ 
tee of Vice-Chancellors and Principals. As this rev¬ 
ersion was not satisfactory, in 1968 and again in 
1970 an independent review was conducted by the 
National Board for Prices and Incomes. During that 
time much of the roie previously undertaken by the 
UGC y'as gradually supplanted by the Department 
of Education and Science. 

The sequence of events now to be described in 
Australia reflects the general direction taken in Bri¬ 
tain, particularly the prominent role played by uni¬ 
versity staff associations and the vice chancellors' 
committees in both countries. Another similarity 
was the establishment by the respective govern¬ 
ments of a permanent formal review mechanism 
outside the area of responsibility of the co¬ 
ordinating agency for universities in each country. 
In Australia, however, there was an added element. 
The tensions between Commonwealth and State 
levels of government complicated the evolutionary 
process. The States were wary of consenting to the 
establishment of a national agency capable of mak- 
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